Reconfigurable and customizable general-purpose circuits for neural networks

ABSTRACT

A reconfigurable neural network circuit is provided. The reconfigurable neural network circuit comprises an electronic synapse array including multiple synapses interconnecting a plurality of digital electronic neurons. Each neuron comprises an integrator that integrates input spikes and generates a signal when the integrated inputs exceed a threshold. The circuit further comprises a control module for reconfiguring the synapse array. The control module comprises a global final state machine that controls timing for operation of the circuit, and a priority encoder that allows spiking neurons to sequentially access the synapse array.

This invention was made with Government support under HR0011-09-C-0002awarded by Defense Advanced Research Projects Agency (DARPA). TheGovernment has certain rights in this invention.

BACKGROUND

The present invention relates to neuromorphic and synaptronic systems,and in particular, reconfigurable and customizable general-purposecircuits for neural networks.

Neuromorphic and synaptronic systems, also referred to as artificialneural networks, are computational systems that permit electronicsystems to essentially function in a manner analogous to that ofbiological brains. Neuromorphic and synaptronic systems do not generallyutilize the traditional digital model of manipulating 0s and 1s.Instead, neuromorphic and synaptronic systems create connections betweenprocessing elements that are roughly functionally equivalent to neuronsof a biological brain. Neuromorphic and synaptronic systems may comprisevarious electronic circuits that are modeled on biological neurons.

In biological systems, the point of contact between an axon of a neuronand a dendrite on another neuron is called a synapse, and with respectto the synapse, the two neurons are respectively called pre-synaptic andpost-synaptic. The essence of our individual experiences is stored inconductance of the synapses. The synaptic conductance changes with timeas a function of the relative spike times of pre-synaptic andpost-synaptic neurons, as per spike-timing dependent plasticity (STDP).The STDP rule increases the conductance of a synapse if itspost-synaptic neuron fires after its pre-synaptic neuron fires, anddecreases the conductance of a synapse if the order of the two firingsis reversed.

BRIEF SUMMARY

Embodiments of the invention describe a reconfigurable neural networkcircuit. In one embodiment, the reconfigurable neural network circuitcomprises an electronic synapse array including multiple synapsesinterconnecting a plurality of digital electronic neurons. Each neuroncomprises an integrator that integrates input spikes and generates asignal when the integrated inputs exceed a threshold. The circuitfurther comprises a control module for reconfiguring the synapse array.The control module comprises a global final state machine that controlstiming for operation of the circuit, and a priority encoder that allowsspiking neurons to sequentially access the synapse array.

These and other features, aspects and advantages of the presentinvention will become understood with reference to the followingdescription, appended claims and accompanying figures.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

FIG. 1 shows a diagram of a neuromorphic and synaptronic networkcomprising a crossbar array of electronic synapses interconnectingelectronic neurons, in accordance with an embodiment of the invention;

FIG. 2A shows a diagram of a neuromorphic and synaptronic circuitcomprising a reconfigurable fully-connected neural network circuit withN neurons and N×N synapses, in accordance with an embodiment of theinvention;

FIG. 2B shows a process for updating/programming synapses in the circuitof FIG. 2A, in accordance with an embodiment of the invention;

FIG. 3 shows a diagram of a digital electronic neuron in the circuit ofFIG. 2A, in accordance with an embodiment of the invention;

FIG. 4 shows a diagram of a crossbar array of electronic synapses anddetails of an electronic synapse at a cross-point junction of thecrossbar array in the circuit of FIG. 2A, in accordance with anembodiment of the invention;

FIG. 5 shows a system timing diagram for neuron and synapse operationsin the circuit of FIG. 2A, in accordance with an embodiment of theinvention;

FIG. 6 shows another system timing diagram for neuron and synapseoperations in the circuit of FIG. 2A, in accordance with an embodimentof the invention;

FIG. 7 shows a system timing diagram for pipelining neuron and synapseoperations in the circuit of FIG. 2A, in accordance with an embodimentof the invention;

FIG. 8 shows learning mode processes based on learning rules for synapseupdates in the circuit of FIG. 2A, in accordance with an embodiment ofthe invention;

FIG. 9 shows further learning mode processes based on learning rules forsynapse updates in the circuit of FIG. 2A, in accordance with anembodiment of the invention;

FIG. 10 shows an example neural network chip architecture based on thecircuit of FIG. 2A, in accordance with an embodiment of the invention;

FIG. 11 shows an example application of a neural network for patternrecognition, in accordance with an embodiment of the invention;

FIG. 12 shows a diagram of a neuromorphic and synaptronic circuitcomprising a crossbar array of multi-bit electronic synapses forinterconnecting digital electronic neurons, in accordance with anembodiment of the invention;

FIG. 13 shows a diagram of a digital electronic neuron in the circuit ofFIG. 12, in accordance with an embodiment of the invention;

FIG. 14 shows a diagram of a crossbar array of electronic synapses anddetails of a multi-bit electronic synapse at a cross-point junction ofthe crossbar array in the circuit of FIG. 12, in accordance with anembodiment of the invention;

FIG. 15 shows a system timing diagram for neuron and synapse operationsin the circuit of FIG. 12, in accordance with an embodiment of theinvention;

FIG. 16 shows learning mode processes based on learning rules forsynapse updates in the circuit of FIG. 12, in accordance with anembodiment of the invention;

FIG. 17 shows further learning mode processes based on learning rulesfor synapse updates in the circuit of FIG. 12, in accordance with anembodiment of the invention; and

FIG. 18 shows a high level block diagram of an information processingsystem useful for implementing one embodiment of the present invention.

DETAILED DESCRIPTION

Embodiments of the invention provide reconfigurable and customizablegeneral-purpose circuits for neural networks. Embodiments of theinvention further provide neuromorphic and synaptronic systems based onthe reconfigurable and customizable general-purpose circuits, includingcrossbar arrays which implement learning rules for re-enforcementlearning.

An embodiment of the reconfigurable and customizable general-purposecircuit provides a system-level computation/communication platformcomprising a neural network hardware chip. The circuit provides areconfigurable compact and low-power digital CMOS spiking networkimplementing binary stochastic STDP on a static random access memory(SRAM) synapse array interconnecting digital neurons. A priority encodersequentially grants array access to all simultaneously spiking neuronsto implement communication of synaptic weights for programming ofsynapses. A global finite state machine module controls timing foroperation of the circuit. Driver module receives digital inputs fromneurons for programming the synapse array using programming phases.Sense amplifiers measure the state of each synapse and convert it tobinary data, representing data stored in the synapse.

Each digital neuron further comprises a learning module including twodigital counters that decay at a pre-specified rate (e.g., about 50 ms)at each timestep and are reset to a pre-defined value when a neuronspiking event occurs. A linear feedback shift register (LFSR) generatesa new random number (e.g., pseudo random number) during everyprogramming phase. A comparator provides a digital signal thatdetermines whether or not a connected synapse is updated (i.e.,programmed). This implements probabilistic updates of synapses accordingto the learning rule specified in the decay rate of the counter.

Timing operations of the general-purpose hardware circuit for neuralnetworks involves a timestep (e.g., based on a biological timestep)wherein within such a timestep multiple neuron spikes and synapseupdates are sequentially handled in a read phase and a write phase,respectively, utilizing a digital clock. Further, variable timesteps maybe utilized wherein the start of a next timestep may be triggered usinghandshaking signals whenever the neuron/synapse operation of theprevious timestep is completed. For external communication, pipeliningis utilized wherein load inputs, neuron/synapse operation, and sendoutputs are pipelined (this effectively hides the input/output operatinglatency).

Referring now to FIG. 1, there is shown a diagram of a neuromorphic andsynaptronic circuit 10 having a crossbar array 12 in accordance with anembodiment of the invention. In one example, the overall circuit maycomprise an “ultra-dense crossbar array” that may have a pitch in therange of about 0.1 nm to 10 μm. The neuromorphic and synaptronic circuit10 includes a crossbar array 12 interconnecting a plurality of digitalneurons 14, 16, 18 and 20. These neurons are also referred to herein as“electronic neurons”. Neurons 14 and 16 are dendritic neurons andneurons 18 and 20 are axonal neurons. Neurons 14 and 16 are shown withoutputs 22 and 24 connected to dendrite paths/wires (dendrites) 26 and28, respectively. Neurons 18 and 20 are shown with outputs 30 and 32connected to axon paths/wires (axons) 34 and 36, respectively.

Neurons 18 and 20 also contain inputs and receive signals alongdendrites, however, these inputs and dendrites are not shown forsimplicity of illustration. Neurons 14 and 16 also contain inputs andreceive signals along axons, however, these inputs and axons are notshown for simplicity of illustration. Thus, the neurons 18 and 20 willfunction as dendritic neurons when receiving inputs along theirdendritic connections. Likewise, the neurons 14 and 16 will function asaxonal neurons when sending signals out along their axonal connections.When any of the neurons 14, 16, 18 and 20 fire, they will send a pulseout to their axonal and to their dendritic connections.

Each connection between dendrites 26, 28 and axons 34, 36 are madethrough a digital synapse device 31 (synapse). The junctions where thesynapse devices are located may be referred to herein as “cross-pointjunctions”. In general, in accordance with an embodiment of theinvention, neurons 14 and 16 will “fire” (transmit a pulse) when theinputs they receive from axonal input connections (not shown) exceed athreshold. Neurons 18 and 20 will “fire” (transmit a pulse) when theinputs they receive from dendritic input connections (not shown) exceeda threshold. In one embodiment, when neurons 14 and 16 fire theymaintain an anti-STFP (A-STDP) variable that decays. For example, in oneembodiment, the decay period may be 50 ms. The A-STDP variable is usedto achieve STDP by encoding the time since the last firing of theassociated neuron. Such STDP is used to control “potentiation”, which inthis context is defined as increasing synaptic conductance. When neurons18, 20 fire they maintain a D-STDP variable that decays in a similarfashion as that of neurons 14 and 16.

A-STDP and D-STDP the variables may decay according to exponential,linear, polynomial, or quadratic functions, for example. In anotherembodiment of the invention, the variables may increase instead ofdecreasing over time. In any event, this variable may be used to achievedendritic STDP, by encoding the time since the last firing of theassociated neuron. Dendritic STDP is used to control “depression”, whichin this context is defined as decreasing synaptic conductance.

An external two-way communication environment may supply sensory inputsand consume motor outputs. Digital neurons implemented usingcomplementary metal-oxide-semiconductor (CMOS) logic gates receive spikeinputs and integrate them. The neurons include comparator circuits thatgenerate spikes when the integrated input exceeds a threshold. In oneembodiment, binary synapses are implemented using transposable 1-bitSRAM cells, wherein each neuron can be an excitatory or inhibitoryneuron. Each learning rule on each neuron axon and dendrite arereconfigurable as described hereinbelow.

FIG. 2A shows a block diagram of a reconfigurable neural network circuit100 implemented as a circuit chip according to an embodiment of theinvention. The circuit 100 includes a synapse array, such as thecrossbar array 12 in FIG. 1, interconnecting multiple digital neurons 5(i.e., N₁, . . . , N_(N)), such as neurons 14, 16, 18, 20 in FIG. 1. Thefully connected synapse array 12 stores the strength of connectionbetween each neuron 5 (e.g., integrate and fire electronic neuron). Eachdigital neuron 5 receives spike inputs from one or more other neuronsand integrates them, such that when the integrated input exceeds athreshold, the digital neuron 5 spikes.

In one embodiment of the circuit 100, neuron operation and parameters(e.g., spiking, integration, learning, external communication) isreconfigurable, customizable, and observable. A priority encoder 101controls access of simultaneously spiking neurons 5 to the crossbararray 12 in a sequential manner. Driver circuits 103 receive digitalinputs from neurons 5 and programs the synapses 31 in the synapse array12 using learning rules. Input pads 105 provide input interfaces to thecircuit 100 and output pads provide output interfaces from the circuit100. A global finite state machine 102 controls timing and operationalphases for operation of the circuit 100. Each synapse interconnects anaxon of a pre-synaptic neuron via a row of the array 12, with a dendriteof a post-synaptic neuron via a column of the array 12. Referring to theprocess 190 in FIG. 2B, according to an embodiment of the invention,within a timestep, the circuit 100 goes through the following sequenceof phases for synapse updating (programming) based on signals from theglobal finite state machine:

-   -   Process block 191: Phase 1—Determine which neurons 5 spiked in a        previous timestep, and reset the potential of those neurons.    -   Process block 192: Phase 2—Perform neuron spiking by pulsing a        row (or axon) of the synapse array 12. Read value of each        synapse 31 in the row and pass the value to a connected neuron        5. Obtain external input for each neuron.    -   Process block 193: Phase 3—Each neuron 5 checks a column (or        dendrite) of the synapse array 12 for synapses 31 in their        “pulsed” state and reads the synapse values, and integrates the        synapse (excitatory/inhibitory) inputs as external input to the        neuron potential.    -   Process block 194: Phase 4—Depending on the time elapsed since        each neuron spiked (fired), probabilistically change a connected        synapse value using a pseudo random number generator such as        said LFSR. Write the new synapse value into the synapse array        12.    -   Process block 195: Phase 5—Determine which neurons will spike        the next timestep by comparing each neuron potential against a        customized threshold.

FIG. 3 shows details of an example implementation of digital neuron 5,according to an embodiment of the invention. In one embodiment, theneuron 5 comprises a reconfigurable digital CMOS circuit device.Specifically, the neuron 5 comprises an integration and spike module 6and a learning module 7. In the integration and spike module 6, amultiplexer circuit 6A is used to select all the inputs arriving at theneuron 5 to integrate to a value held at an adder circuit 6B. The valuein the adder circuit 6B represents the potential of the neuron 5 (e.g.,voltage potential V based on accumulated input spikes). A comparatorcircuit 6C is used to check if the current value in the adder 6B isabove a threshold value. The output of the comparator 6C is used tosignal neuron spiking. This spike signal is then sent to the priorityencoder 101 which then grants the neuron 5 access to the crossbarsynapse array 12 in a sequential manner.

The learning module 7 includes digital counters 7A and 7B, which decayat a pre-specified rate each time step and are reset to a pre-definedvalue when the neuron spikes. A LFSR 7C generates sequences that aremaximally random. During every synapse programming phase, the LFSR 7Cgenerates a new random number. A comparator circuit 7D compares therandom number with a counter value (i.e., from counters 7A and 7B via amultiplexer 7E) to provide a digital signal that determines whether ornot a synapse 31 is updated (i.e., programmed). As such, synapses 31 areupdated probabilistically according to a learning rule specified in thedecay rate of a counter (i.e., counters 7A and 7B). In one embodiment,the causal counter 7B is used for pre-synaptic updates, and theanti-causal counter 7A is used for post-synaptic update (pre-synapticand post-synaptic updates may utilize different learning rules).

Table 1 below shows an example neuron specification in conjunction withthe circuits in FIGS. 2A and 3, according to an embodiment of theinvention.

TABLE 1 Neuron specification Number of bits for 8 bits + 4 overflow + 4underflow bits neuron potential Number of bits for 10 least significantbits (LSBs) derived from LFSR counter a 15 bit LFSR Number of bits for 8bits tau counter Number of LFSRs One (used twice to generate the randomper neuron number for causal and anti-causal update) Number of tau Two(one for causal and one for anti-causal). counters per neuron All neuronconfigurations maybe customized as needed. One exemplary configurationis listed in entry below. Configuration 1 bit to turn on and offlearning 1 bit to specify if neuron is inhibitory or excitatory (bit isstored in priority encoder)

Table 2 below shows an example neuron configuration for control andobservability in conjunction with the circuits in FIGS. 2A and 3,according to an embodiment of the invention. All scan configurations(for control and observability) could be customized as needed. In Table2 “b” means bits.

TABLE 2 Neuron reconfiguration/observation Reconfiguration Control (ScanIn) Observation (Scan Out) 8 b: Excitatory weight (s+) 1 b:Excite/Inhibit input (SA out) 8 b: Inhibitory weight (s−) 16 b: Neuronpotential (Vn) 8 b: Leak parameter (λ) 1 b: Spike signal (θ compare out)8 b: External input weight (s_(ext)) 10 b: LFSR output 8 b: Threshold 8b*2: Tau counter output 1 b: Learning enable 1 b: STDP comparator output28 b: Learning parameters—8 b*2, 3 b*2, 6 b 4 b: Represent 16 learningmodes Total: 73 b Total: 45 b

As noted, in one embodiment each synapse interconnects an axon of apre-synaptic neuron with a dendrite of a post-synaptic neuron. As such,in one embodiment, the circuit 100 comprises a first learning module foran axonal, pre-synaptic, neuron, and a second learning module for adendritic, post-synaptic neuron, such that each of the learning modulesis reconfigurable independent of the other.

FIG. 4 shows an example implementation of the synapse crossbar array 12of FIG. 2A, according to an embodiment of the invention. The synapses 31are binary memory devices, wherein each synapse can have a weight “0”indicating it is non-conducting, or a weight “1” indicating it isconducting. In one embodiment, a synapse 31 comprises a transposableSRAM cell (e.g., transposable 8-T SRAM cell). The binary synapses 31 areupdated probabilistically (e.g., using random number generators inneurons 5, as described further above). The crossbar array 12 cancomprise a N×N transposable SRAM synapse array implementing a fullyconnected crossbar for N digital neurons 5 (e.g., N=16). A transposablecell 31 is utilized for pre-synaptic (row) and post-synaptic (column)synapse updates. WL stands for wordlines and BL stands for bitlines asfor memory arrays. For transposability, WL, BL, BL (inversion of BL) areresponsible for the row updates, and WL_(T), BL_(T), BL _(T) areresponsible for the column updates.

FIG. 5 shows an example system timing diagram 120 for neuron and synapseoperation for the circuit 100 in conjunction with FIGS. 2A, 3 and 4,according to an embodiment of the global finite state machine 102. Asillustrated in FIG. 5, sequential operation of neurons 5 is in atimestep implemented utilizing phases/cycles 122 of a digital clock(hardware (HW) clock), such as may be provided by the global finitestate machine 102. All spiking neurons 5 first complete theircommunication in n cycles 123, and the updates for the synapses 31 ontheir axons and dendrites are completed in 2n cycles 124. A horizontalupdate (axonal synapse update in array 12) is for updating weights ofsynapses in a row of the crossbar array 12, and a vertical update(dendritic synapse update in array 12) is for updating weights ofsynapses in a column of the crossbar array 12.

FIG. 6 shows another system timing diagram 125 for neuron and synapseoperation for the circuit 100 in conjunction with FIGS. 2A, 3 and 4, forvariable timesteps 126, according to an embodiment of the global finitestate machine 102. The circuit 100 loads input data, performsneuron/synapse operations and sends out output date. For a variabletimestep operation mode, when the circuit 100 has completed neuron andsynapse operations before end of an allocated timestep, a neuron/synapseoperation completion signal is generated to indicate that a next timestep can begin without idling for current timestep to end. This providesoverall faster learning time for the circuit 100. For a fixed timestepoperation mode, the completion signal is not generated. Each software(SW) clock has a number of HW clocks.

SW clock corresponds to a biological timestep. Within one biologicaltimestep, multiple operations are performed, including digital neuronpotential integration, learning computation, synapse update, etc. Suchoperations may be performed in a sequential and pipelined manner,wherein each said timestep is divided into multiple (e.g., hundreds) ofHW clock cycles, as shown by example in FIGS. 5-6. The HW clock cyclesgovern the digital neuron operations and synapse array updates asdisclosed herein.

FIG. 7 shows a system timing diagram 130 for pipelining neuron andsynapse operation for the circuit 100 in conjunctions with FIGS. 2A, 3and 4, according to an embodiment of the global finite state machine102. The pipelined operations of circuit 100 include three phases: loadinput, neuron/synapse operations (crossbar operation) and send output.In one example, a single clock of e.g. 1 MHz frequency is utilized forthe HW clocking cycles 122, in each timestep 132, for data in/outlatches, neuron latches, and other latches, and for clock gating eachpipeline phase separately.

According to embodiments of the invention, the learning rules can bereconfigured depending on the algorithm or a certain application and arenot limited to STDP learning rules. For example, anti-STDP, Hebbian,anti-Hebbian, and any other types of learning rules may be utilized.

FIG. 8 shows learning mode processes 141, 143, 145 for learning rulesSTDP, Anti-STDP (A-STDP) and Hebbian, respectively, in the circuit 100without constant, according to an embodiment of the invention. Thelearning mode processes are performed in conjunction with neuron circuit5 in FIG. 2A for probabilistic synapse updates. No synapse updates areperformed for a non-learning mode. The τ (tau) counter value of adigital neuron 5 decreases as time elapses since the last spike. For asingle-bit synapse update, when τ reaches 0, a constant (const) may beinvolved in the learning process. A synapse update may occur regardlessof the value of τ.

Referring to FIG. 9, when a constant is involved in the learningprocess, once τ reaches 0, the constant is compared with a random numberfrom LFSR and update is performed with a certain probability. FIG. 9shows learning mode processes 147, 148, 149 for learning rules STDP,Anti-STDP (A-STDP) and Hebbian, respectively, in the circuit 100 withconstant, according to an embodiment of the invention. The learning modeprocesses are performed in conjunction with neuron circuit 5 in FIG. 2Afor probabilistic synapse updates.

TABLE 3 Learning modes Learning STDP, anti-STDP, Hebbian, anti-Hebbianmodes Learning 8 bits (for causal) + 8 bits (for anti-causal), foramplitude curve 3 bits (for causal) + 3 bits (for anti-causal), fordecay rate specifi- 6 bits for constant level comparison (same forcausal and cation anti-causal) 4 bits to specify learning mode

Left and right learning curves of each learning mode in FIG. 9 canindependently select whether a constant is used or not.

FIG. 10 shows an example chip architecture 160 based on the circuit 100,according to an embodiment of the invention. The synapse crossbar arraycomprises a N×N array of synapses 31 for N neurons 5, comprising twoconnected layers E1 and E2 of electronic neurons including excitatoryneurons (Ne), and inhibitory neurons (Ni). The global finite statemachine 102 includes a bit that sets the chip either in E1-E2 mode orfully connected array mode. During an initialization phase, the weightof synapses 31 in a diagonal block are set to 0 (as shown in top part ofFIG. 11), and are never allowed to change. Each neuron 5 has 1 bit tospecify if it is an E1 neuron or an E2 neuron. When a neuron spikes, aflag is set in the priority encoder 101 to indicate if the spikingneuron is an E1 neuron or an E2 neuron. This information is used by theother neurons for synapse update. During an update (learning) phase, asynapse 31 is updated only if it is at the intersection (cross-pointjunction in an array 12) of an E1 neuron and an E2 neuron. Table 4 belowshows example excitatory and inhibitory neuron configuration for E1-E2mode, according to an embodiment of the invention.

TABLE 4 Excitatory and inhibitory neuron for E1-E2 Number of excitatoryX (any number, X + H < N) neurons in E1-E2 Number of inhibitory H (anynumber, X + H < N) neurons in E1-E2 Excitatory neuron Both Ne and Ni usesame hardware infrastructure, (Ne) and inhibitory but, the parameterscan be set/reconfigured to neuron (Ni) behavior provide differentbehaviors (for Ne add to, for Ni subtract from, the neuron potentialwhen a spike arrives). Input and output Axon of each Ni will have ‘X’ ONsynapses and neurons of inhibitory dendrite of each Ni will have ‘X/2’ON synapses neuron for E1-E2 connecting to the E2 neurons. The synapsesare not plastic. This connectivity is initialized at the beginning ofthe chip operation. (In general, however, Ni may have the exact samebehaviors as Ne, including having plastic synapses.) Output ofinhibitory Ni output is locally fed into the synapse crossbar. neuron?When a Ni fires/spikes, a global flag is set, to indicate that thecurrent incoming signal at the input ports of receiving neurons has tobe subtracted.

If a specified learning rule (i.e., E1-E2) uses only less than Nneurons, the remaining neurons that are not participating in E1-E2 areinactive. For the inactive neurons, the synapses on dendrites and axonsof inactive neurons are initialized to 0, external input for inactiveneurons in every cycle is 0, and learning is disabled using aconfiguration bit.

FIG. 11 shows an example application of the chip 100 based on thearchitecture 160 for pattern recognition in an input image, according toan embodiment of the invention. The chip 100 performs non-linear patternclassification, by interacting with the external environment in theforms of digital spike inputs and outputs. The neurons 5 in the circuit100 simulate retinal neurons and during learning phase based on theinput pattern the neurons 5 spike and synapses 31 learn the input imagepixel pattern. Not all the pixels that are lighted have to come in atthe same time. The input image comes in as frames, and the circuit 100integrates the frames together as one object. Then, in a recall phase,when a corrupted frame is provided to the circuit 100, the full patternis recalled (i.e., pattern recognition). Any data input stream, can belearned, classified, and recalled. As such, in a learning phase, thearchitecture 160 learns correlations in spatio-temperal patterns andclassification of said patterns. Once the learning is completed, thecircuit can predict and complete incomplete patterns in a recall phase.

The digital circuit 100 also provides fine-grain verification betweenhardware and software for every spike, neuron state, and synapse state.For the E1-E2 configuration, comparing the spike outputs andneuron/synapse states of hardware and software simulation, a one-to-oneequivalence is achieved for the full-length simulation of 15,000(biological) timesteps. The circuit 100 can be mounted on a stand-alonecard interacting with the external environment such as a computer orother computing platform (e.g., a mobile electronic device).

In another embodiment, the present invention provides a neuronal circuitcomprising multi-bit transposable crossbar array of SRAM synapses forinterconnecting digital neurons. The circuit provides general-purposehardware that enhances the pattern classification capability of aspiking neural network by interacting with the external environment inthe forms of digital spike inputs and outputs. In one implementation,the circuit comprises a low-power digital CMOS spiking neural networkthat is reconfigurable, to implement stochastic STDP on multi-bitsynapse array for interconnecting digital neurons, with improvedlearning capability.

In one embodiment, multi-bit (m-bit) synapses are implemented usingtransposable SRAM cells which can store a value from 0 to 2m−1,representing a fine-grain connection between every neuron connection ina multi-bit synapse array. In one implementation, the values in therange 0 to 2m−1 represents the level of conductivity of a synapse. Apriority encoder sequentially allows array access to all simultaneouslyspiking neurons to implement communication of synaptic weights andprogramming of synapses. A global finite state machine module controlsthe entire operation of the chip including the multi-bit synapse array.Driver modules receive digital inputs from neurons and program themulti-bit synapse array.

Each digital neuron comprises a learning module including two countersthat decay at every pre-specified number of timesteps and are reset to apre-defined value when a neuron spiking event occurs. In a synapseupdate phase, the learning module reads the existing multi-bit synapsevalue from the synapse array, adds or subtracts the decay counter valueto the value read from the array, and updates the modified new multi-bitvalue to the synapse array.

Timing operations in the update phase with multi-bit synapse array aresuch that multiple read and write operations can occur in the synapseupdate phase in a timestep. To reduce read/write latency to the synapsearray from each neuron, the read and write operations are interleavedsuch that every hardware cycle is performing either a synapse read orwrite, increasing the overall throughput.

FIG. 12 shows a block diagram of a reconfigurable neural network circuit200 implemented as a circuit chip including a multi-bit transposablecrossbar array of SRAM synapses interconnecting digital neurons,according to an embodiment of the invention. The circuit 200 includes acrossbar array 212 of multi-bit synapses 131 (FIG. 14) interconnectingmultiple digital neurons 15 (i.e., N₁, . . . , N_(N)). The digitalneurons 15 associated with multi-bit SRAM array 212 provide learningcapability in a spiking neural network. Multi-bit synapses provide noisetolerance. Every neuron operation and parameter (spiking, integration,learning, external communication) intended for multi-bit synapses isreconfigurable, customizable, and observable. The circuit 200 achievesimprovement in learning time, which enables more complicated patternrecognition.

The multi-bit synapse array 212 stores the strength of connectionbetween each neuron 15 in a fine-grain value between 0 and 2^(m)−1.Digital neurons 15 receive multi-bit spike inputs and integrate them,such that in each neuron when the integrated input exceeds a threshold,the neuron spikes. In one example, within a timestep, neuron and synapseoperations in the circuit 200 go through the following sequence ofsynapse updating (programming):

-   -   Phase 1: Determine which neurons 15 spiked in the previous        timestep, and reset the potential of those neurons.    -   Phase 2: Implement neuron spiking by pulsing a row (or axon) of        the crossbar array 212. Read value of each synapse 131 and pass        it to a connected neuron 15. Obtain external input for each        neuron.    -   Phase 3: Each neuron 15 checks its dendrites for synapses 131 in        their “pulsed” state and reads their multi-bit values. Integrate        the multi-bit synapse (excitatory/inhibitory) input, as external        input to the neuron potential.    -   Phase 4: Read the existing multi-bit synapse value from the SRAM        array 212. Modify the synapse value by adding or subtracting the        tau counter value, depending on the learning rule/mode. Write        the new multi-bit synapse value into the SRAM array 212.    -   Phase 5: Determine which neurons will spike the next time step        by comparing each neuron potential against the customized        threshold.

FIG. 13 shows details of an example implementation of a digital neuron15 in the circuit 200, according to an embodiment of the invention.Multi-bit input and output channels exist between each neuron 15 and themulti-bit synapse array 212. The neuron 15 comprises an integration andspike module 56 and a learning module 57. A multi-bit value is passed onfrom the synapse array 212 to the neuron 15 for integration in the spike(read) phase. In the integration and spike module 56, a multiplexercircuit 56A is used to select all the inputs arriving at the neuron 15to integrate to a value held at an adder circuit 56B. The value in theadder circuit 56B represents the potential of the neuron 15 (e.g.,voltage potential V based on accumulated input spikes). A comparatorcircuit 56C is used to check if the current value in the adder 56B isabove a threshold value. The output of the comparator 56C is used tosignal neuron spiking. This spike signal is then sent to the priorityencoder 101 which then grants the neuron 15 access to the crossbar array212 in a sequential manner.

The learning module 57 includes digital counters 57A and 57B, whichdecay at a pre-specified rate each time step and are reset to apre-defined value when the neuron 15 spikes. In the update (write)phase, the learning module goes through the process of read-modify-writeincluding: reading the existing multi-bit synapse value from the synapsearray 212, adding or subtracting the decay counter value to the valueread from the array 212, and updating the modified new multi-bit valueto the synapse array 212. As such, the synaptic strength (multi-bitsynapse value) of 131 synapses between neurons 15 are strengthened orweakened every time-step according to the time elapsed since a neuronspiked. The adder 57D adds (or subtracts) the τ counter value to (orfrom) the current synapse value. Compared to neuron 5 in FIG. 3, theneuron 15 does not utilize weighting factors to the input of themultiplexer 56, and does not utilize a probability generator.

FIG. 14 shows an example implementation of the synapse crossbar array212 of FIG. 12, according to an embodiment of the invention. Thesynapses 131 are multi-bit memory devices. In one embodiment, a synapse131 comprises m transposable SRAM cells (e.g., transposable 8-T SRAMcell). The synapses 131 are updated as described further above. Multiple(m) transposable SRAM cells 31 are used in each multi-bit synapse 131for pre-synaptic (row) and post-synaptic (column) update in the array212. On each horizontal (row) and vertical (column) direction, asingle-bit cell 31 uses a pair of bit lines, such that an multi-bitsynapse has m pairs of bit lines, and the entire multi-bit synapsevalues are written at once (i.e., using only one word line). Thecrossbar array 212 can comprise a N×N transposable SRAM synapse array ofmulti-bit synapse cells 131 implementing a fully connected crossbar forN digital neurons 15.

FIG. 15 shows an example system timing diagram 121 for neuron operationfor the circuit 200 in conjunction with FIGS. 12-14, according to anembodiment of the global finite state machine 102. As illustrated inFIG. 15, sequential operation of neurons 15 is performed within atimestep, utilizing phases/cycles 122 of a digital clock (HW clock),which is governed by the global finite state machine 102. In an updatephase, a read-modify-write process is performed, and the synapseread/write operations are interleaved to maximize throughput. Allspiking neurons 15 first complete their communication in n cycles 136,and the updates for the synapses 131 on the axons and dendrites arecompleted in 4n cycles 137 for the crossbar array 212.

In one embodiment, the circuit 200 can operate in fixed time step andvariable time step modes, similar to that described for circuit 100further above in relation to FIG. 6. In one embodiment, the circuit 200can operate in pipeline fashion, similar to that described for circuit100 further above in relation to FIG. 7.

In the circuit 200, learning rules for synapses are based entirely oncausality (no probabilistic synapse updated). FIG. 16 shows learningmode processes 142, 144, 146 for learning rules STDP, Anti-STDP (A-STDP)and Hebbian, respectively, in the circuit 200 for 4-bit synapses 131without constant, according to an embodiment of the invention. The valueof S is bounded between 0 and 2^(m)−1. The learning mode processes areperformed in conjunction with neuron circuit 15 in FIG. 12. The τ (tau)counter value of a digital neuron 15 decreases as time elapses since thelast spike. For a multi-bit synapse update, when τ reaches 0, a constant(const) may be involved in the learning process. When a constant is notinvolved in the learning process, the synapse update is performed asshown in FIG. 16.

Referring to FIG. 17, when a constant is involved in the learningprocess, the constant is added to (or subtracted from) the currentmulti-bit synapse value once τ reaches 0. FIG. 17 shows learning modeprocesses 151, 153, 155 for learning rules STDP, Anti-STDP (A-STDP) andHebbian, respectively, in the circuit 200 with constant, according to anembodiment of the invention. The learning mode processes are performedin conjunction with neuron circuit 15 in FIG. 12 for synapse updates.Left and right learning curves of each mode can independently selectwhether a constant is used or not. The value of S is bounded between 0and 2^(m)−1.

Table 5 below shows an example neuron configuration for control andobservability in conjunction with the circuits in FIGS. 12 and 13,according to an embodiment of the invention. All scan configurations(for control and observability) could be customized as needed. In Table5 “b” means bits.

TABLE 5 Neuron reconfiguration/observability Reconfiguration Control(Scan In) Observability (Scan Out) 8 b: Leak parameter (λ) 1 b:Excite/Inhibit input (SA out) 8 b: External input weight (s_(ext)) 16 b:Neuron potential (Vn) 8 b: Threshold 1 b: Spike signal (θ compare out) 1b: Learning enable 4 b*2: Tau counter output 18 b: Learning parameters—4b*2, 3 b*2, 6 b*2: Slope counter output 4 b 4 b: Represent 16 learningmodes 1 b: STDP comparator output Total: 47 b Total: 39 b

FIG. 18 is a high level block diagram showing an information processingcircuit 300 useful for implementing one embodiment of the presentinvention. The computer system includes one or more processors, such asprocessor 302. The processor 302 is connected to a communicationinfrastructure 304 (e.g., a communications bus, cross-over bar, ornetwork).

The computer system can include a display interface 306 that forwardsgraphics, text, and other data from the communication infrastructure 304(or from a frame buffer not shown) for display on a display unit 308.The computer system also includes a main memory 310, preferably randomaccess memory (RAM), and may also include a secondary memory 312. Thesecondary memory 312 may include, for example, a hard disk drive 314and/or a removable storage drive 316, representing, for example, afloppy disk drive, a magnetic tape drive, or an optical disk drive. Theremovable storage drive 316 reads from and/or writes to a removablestorage unit 318 in a manner well known to those having ordinary skillin the art. Removable storage unit 318 represents, for example, a floppydisk, a compact disc, a magnetic tape, or an optical disk, etc. which isread by and written to by removable storage drive 316. As will beappreciated, the removable storage unit 318 includes a computer readablemedium having stored therein computer software and/or data.

In alternative embodiments, the secondary memory 312 may include othersimilar means for allowing computer programs or other instructions to beloaded into the computer system. Such means may include, for example, aremovable storage unit 320 and an interface 322. Examples of such meansmay include a program package and package interface (such as that foundin video game devices), a removable memory chip (such as an EPROM, orPROM) and associated socket, and other removable storage units 320 andinterfaces 322 which allow software and data to be transferred from theremovable storage unit 320 to the computer system.

The computer system may also include a communication interface 324.Communication interface 324 allows software and data to be transferredbetween the computer system and external devices. Examples ofcommunication interface 324 may include a modem, a network interface(such as an Ethernet card), a communication port, or a PCMCIA slot andcard, etc. Software and data transferred via communication interface 324are in the form of signals which may be, for example, electronic,electromagnetic, optical, or other signals capable of being received bycommunication interface 324. These signals are provided to communicationinterface 324 via a communication path (i.e., channel) 326. Thiscommunication path 326 carries signals and may be implemented using wireor cable, fiber optics, a phone line, a cellular phone link, an RF link,and/or other communication channels.

In this document, the terms “computer program medium,” “computer usablemedium,” and “computer readable medium” are used to generally refer tomedia such as main memory 310 and secondary memory 312, removablestorage drive 316, and a hard disk installed in hard disk drive 314.

Computer programs (also called computer control logic) are stored inmain memory 310 and/or secondary memory 312. Computer programs may alsobe received via communication interface 324. Such computer programs,when run, enable the computer system to perform the features of thepresent invention as discussed herein. In particular, the computerprograms, when run, enable the processor 302 to perform the features ofthe computer system. Accordingly, such computer programs representcontrollers of the computer system.

From the above description, it can be seen that the present inventionprovides a system, computer program product, and method for implementingthe embodiments of the invention. References in the claims to an elementin the singular is not intended to mean “one and only” unless explicitlyso stated, but rather “one or more.” All structural and functionalequivalents to the elements of the above-described exemplary embodimentthat are currently known or later come to be known to those of ordinaryskill in the art are intended to be encompassed by the present claims.No claim element herein is to be construed under the provisions of 35U.S.C. section 112, sixth paragraph, unless the element is expresslyrecited using the phrase “means for” or “step for.”

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the invention. Asused herein, the singular forms “a”, “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises”and/or “comprising,” when used in this specification, specify thepresence of stated features, integers, steps, operations, elements,and/or components, but do not preclude the presence or addition of oneor more other features, integers, steps, operations, elements,components, and/or groups thereof.

The corresponding structures, materials, acts, and equivalents of allmeans or step plus function elements in the claims below are intended toinclude any structure, material, or act for performing the function incombination with other claimed elements as specifically claimed. Thedescription of the present invention has been presented for purposes ofillustration and description, but is not intended to be exhaustive orlimited to the invention in the form disclosed. Many modifications andvariations will be apparent to those of ordinary skill in the artwithout departing from the scope and spirit of the invention. Theembodiment was chosen and described in order to best explain theprinciples of the invention and the practical application, and to enableothers of ordinary skill in the art to understand the invention forvarious embodiments with various modifications as are suited to theparticular use contemplated.

1. A reconfigurable neural network circuit, comprising: an electronicsynapse array comprising multiple digital synapses interconnecting aplurality of digital electronic neurons, wherein each neuron comprisesan integrator that integrates input spikes and generates a spike signalwhen the integrated input spikes exceed a threshold; and a controlmodule for reconfiguring the synapse array, the control modulecomprising a global final state machine that controls timing foroperation of the circuit, and a priority encoder that allows spikingneurons among said neurons, to sequentially access the synapse array. 2.The network circuit of claim 1, wherein the circuit provides one ofSTDP, anti-STDP, Hebbian and anti-Hebbian learning rules on the synapsearray.
 3. The network circuit of claim 2, wherein the synapse arraycomprises a random access memory synapse array, each neuron comprises areconfigurable digital CMOS device and the circuit provides binarystochastic spike timing dependent plasticity using the random accessmemory synapse array.
 4. The network circuit of claim 1, furthercomprising: driver modules that receive digital inputs from said neuronsand program the synapse array based on the digital inputs and learningrules; and sense amplifiers that measure the state of each synapse andconvert it to binary data.
 5. The network circuit of claim 4, wherein:each neuron comprises a learning module including digital counters thatdecay at a pre-specified rate during each timestep, and are reset to apre-defined value when a neuron spiking event occurs.
 6. The networkcircuit of claim 5, wherein: the priority encoder allows spiking neuronsamong said neurons to sequentially access the synapse array, providingcommunication of synaptic weights and programming of said synapses. 7.The network circuit of claim 6, wherein: each synapse interconnects anaxon of a pre-synaptic neuron with a dendrite of a post-synaptic neuron,the circuit further comprising a first learning module for an axonal,pre-synaptic neuron, and a second learning module for a dendritic,post-synaptic neuron, such that each learning module is reconfigurableindependent of the other; and neuron parameters including spiking,integration, learning and communication for each neuron arereconfigurable using reconfiguration input controls.
 8. The networkcircuit of claim 7, wherein: during a timestep, multiple neuron spikesare sequentially handled in a read phase and synapse updates aresequentially handled in a write phase, utilizing cycles generated by adigital clock, wherein a timestep is divided into multiple digital clockcycles.
 9. The network circuit or claim 8, wherein: the start of asubsequent timestep is triggered using handshaking signals when neuronand synapse operations of a previous timestep are completed.
 10. Thenetwork circuit of claim 9, wherein: input operations, neuron andsynapse operations, and output operations are pipelined.
 11. The networkcircuit of claim 7, wherein the learning module further comprises alinear feedback shift registers for probabilistically updating saidsynapses according to learning rules based on a decay rate of said oneor more digital counters.
 12. The network circuit of claim 11, wherein:each synapse comprises a binary synapse including a transposable 1-bitstatic random access memory cell; and the linear feedback shift registergenerates a new random number during every phase for a programmingsynapse.
 13. The network circuit of claim 12, wherein: the learningmodule further comprises a comparator that compares the random numberwith a spike counter, to generate a digital signal for programming aconnected synapse by a probabilistic update according to a learning rulespecified in a decay rate of the counter.
 14. The network circuit ofclaim 7, wherein in a learning phase, the circuit learns correlations inspatio-temperal patterns and classification of said patterns in thesynapse array, and in a recall phase the reconfigurable neural networkcircuit predicts and completes incomplete patterns.
 15. The networkcircuit of claim 7, wherein the circuit is reconfigurable for learningrules including STDP, anti-STDP, Hebbian and anti-Hebbian.
 16. Areconfigurable neural network circuit, comprising: an electronic synapsearray comprising multiple digital synapses interconnecting a pluralityof digital electronic neurons, wherein each neuron comprises anintegrator that integrates input spikes and generates a spike signalwhen the integrated input spikes exceed a threshold; and a controlmodule for reconfiguring the synapse array, the control modulecomprising a global final state machine that controls timing foroperation of the circuit, and a priority encoder that allows spikingneurons among said neurons, to sequentially access the synapse array;wherein each synapse comprises a multi-bit synapse including multipletransposable 1-bit static random access memory cells.
 17. The networkcircuit of claim 16, wherein each neuron comprises a reconfigurabledigital CMOS circuit device and the circuit provides multi-bitdeterministic spike timing dependent plasticity using the synapse array.18. The network circuit of claim 16, further comprising: driver modulesthat receive digital inputs from said neurons and program the synapsearray based on the digital inputs and learning rules; and senseamplifiers that measure the state of each synapse and convert it todata.
 19. The network circuit of claim 18, wherein: each neuroncomprises a learning module including digital counters that decay at apre-specified rate during each timestep, and are reset to a pre-definedvalue when a neuron spiking event occurs.
 20. The network circuit ofclaim 19, wherein: the priority encoder allows spiking neurons amongsaid neurons to sequentially access the synapse array, providingcommunication of synaptic weights and programming the synapses.
 21. Thenetwork circuit of claim 20, wherein: each synapse interconnects an axonof a pre-synaptic neuron with a dendrite of a post-synaptic neuron, thecircuit further comprising a first learning module for an axonal,pre-synaptic neuron, and a second learning module for a dendritic,post-synaptic neuron, such that each learning module is reconfigurableindependent of the other; and neuron parameters including spiking,integration, learning and communication for each neuron arereconfigurable using reconfiguration input controls.
 22. The networkcircuit of claim 21, wherein: during a timestep, multiple neuron spikesare sequentially handled in a read phase and synapse updates aresequentially handled in a write phase, utilizing cycles generated by adigital clock, wherein a timestep is divided into multiple digital clockcycles.
 23. The network circuit of claim 19, wherein each synapsecomprises m transposable static random access memory cells that store amulti-bit value, representing a fine grain connection between everyneuron connection.
 24. The network circuit of claim 23, wherein: in anupdate phase, the neuron reads an existing multi-bit synapse value fromthe synapse array, adds or subtracts the decay counter value to thevalue read from the synapse to generate a new multi-bit value, andupdates the synapse with the new multi-bit value, such that the synapseupdate operation is performed in the update phase when a neuron spikes.25. The network circuit of claim 24, wherein multiple read and writeoperations are performed in an update phase within a timestep, and theread and write operation are interleaved such that in every hardwarecycle the circuit performs either a synapse read or write.
 26. Thenetwork circuit of claim 21, wherein in a learning phase, the circuitlearns correlations in spatio-temperal patterns and classification ofsaid patterns in the synapse array, and in a recall phase thereconfigurable neural network circuit predicts and completes incompletepatterns.
 27. The network circuit of claim 21, wherein the circuit isreconfigurable for learning rules including STDP, anti-STDP, Hebbian andanti-Hebbian.
 28. A method for producing spike-timing dependentplasticity in a reconfigurable neural network circuit, the methodcomprising: integrating input spikes in an integrate and fire electronicdigital neuron, and upon the integrated input spikes exceeding athreshold, sending a spiking signal to digital synapses interconnectingthe neuron to other neurons via a synapse array, wherein the synapsearray comprises multiple digital synapses interconnecting a plurality ofintegrate and fire electronic digital neurons, each synapseinterconnecting an axon of a pre-synaptic neuron with a dendrite of apost-synaptic neuron; providing control signals to reconfigurationinputs of a neuron to reconfigure neuron operation parameters includingone or more of: spiking, integration, learning and communication; andgenerating signals for phased operation of the neurons and updating thesynapses, and sequential access to the synapse array by spiking.
 29. Themethod of claim 28, further comprising: determining which neurons amongsaid neurons spiked in a previous timestep, and resetting the potentialof the spiking neurons; performing neuron spiking by pulsing an axon rowin the synapse array, reading value of each synapse, and passing thevalue to a connected neuron to obtain an external input for each neuron;each neuron checking a dendrite column in the synapse array for synapsesin their pulsed state, reading the synapse values, and integrating eachsynapse input as external input to the neuron potential; depending onthe time elapsed since each neuron fired, probabilistically changing thesynapse value and writing a new synapse value into the synapse array;and determining which neurons will spike the next timestep by comparingeach neuron potential against a threshold.
 30. The method of claim 29,further comprising: upon receiving an external spike signal in eachneuron, updating a digital counter representing electric potential of amembrane of the neuron based on the external spike signal; decaying theelectric potential of the membrane based on a leak rate; upon theelectric potential of the membrane exceeding a threshold, generating aspike signal.